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ABSTRACT 
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are arrayed against a range of parties at interest, including 
individual students, faculty, institutions, external constituents, 
and the public. Information is provided on benefits of assessment by 
identified parties at interest, dollar costs of assessment, and some 
externalities of assessment. Policy alternatives include: statewide 
testing of basic skills; and requiring institutions to undertake 
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BENEFITS AND COSTS OF ASSESSMENT 
IN HIGHER EDUCATION: 
A FRAMEWORK FOR POLICY CHOICE 
AND COMPARISON 



PETER T. EWELL 



INTRODUCTION 



Over the past three years, calls to 
investigate and improve the quality of 
higher education have rapidly escalated. 
Among the most visible and insistent 
have been initiatives aimed at deter- 
mining the outcomes of undergraduate 
instruction. The resulting "assessment 
movement" has become a national 
phenomenon-embracing activities as 
diverse as state programs to determine 
the basic skills of entering freshmen, 
state and eccreditation-based efforts to 
encourage colleges and universities to 
broadly investigate undergraduate 
outcomes, and a range of institutional 
efforts including testing, student surveys, 
and program evaluation. 

Despite considerable activity, evidence 
about the effectiveness of assessment 
efforts remains scanty. Partly this is 
because most are new and have 
consequently had little opportunity to 
show results. Partly it is because 
documentation of both the benefitj and 
the costs of "assessment" remains 
elusive. 

Within these substantial limits, the 
purpose of this paper is to analyze and 
discuss what is currently known about 
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the benefits and costs of assessment 
programs from tlie general perspective 
of state liiglier education policy. In 
drawing upon national experience in 
these two areas, however, appropriate 
caveats and adjustments may be 
necessary to tailor the analysis to 
specific economic, governance, and 
political conditions in the state of 
Arizona. 

ASSESSMENT IN HIGHER 
EDUCATION: 

BACKGROUND 

Current interest in assessing the 
outcomes of undergraduate education is 
rooted in two simultaneous but 
independent developments. The first is 
reemerging concern about the structure 
and content of the undergraduate 
curriculum. Partly fueled by national 
investigations of deficiencies In 
elementary and secondary education (for 
example the Nation at Risi< report issued 
by the U.S. Dopartment cf Education in 
1983), a number of reform proposals 
directea toward improving under- 
graduate education emerged in 1984-85. 
Among thern were Involvement in 
Learning (the report of the National 
Institute for Education's Study Group on 
the Conditions of Excellence in American 
Higher Education), Integritv in the 
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College Curriculum (issued by the 
American Association of Colleges), To 
Reclaim a Legacy (a report of the 
National Endowment for the Humanities), 
and Access to Quality Undergraduate 
Education (issued by the Southern 
Regional Education Board). The 
consensus of these reports-originating 
for the most part from within the 
academic establishment-was that 
undergraduate instruction was in need of 
considerable overhaul. Particularly 
recommended were: 

(1) greater curricular coherence with 
respect to content and structure, 

(2) higher expectations and standards, 

(3) greater opportunity for active 
student involvement in learning, 

(4) improved capabilities to identify 
and remediate learning deficiencies 
in incoming students, and 

(5) better mechanisms for providing 
"feedback" on learning to students, 
faculty and administrators in order 
to guide improvements. 

It is important to note that only the last 
two of these embrace "assessment" as it 
is currently understood. And both go 
considerably beyond the notion of 
collecting information on undergraduate 
student performance by stressing the 
actual uses to which such information 
will be put. 

A second stimulus for "assessment" is 
increased pressure for accountability 
with regard to public funds invested in 
higher education. One component 
concern is the escalating cost and 
complexity of public postsecondary 
education. Both legislators and 
governors are becoming increasingly 



sensitive (especially in tight budget 
years) to the fact that they know very 
little about the actual impacts of 
investments that can consume 
substantial proportions of state general 
revenue funds each year. At the same 
time, higher education leaders have 
become more sophisticated in their 
claims about public benefit. Links 
between proposed investments in public 
higher education and statewide 
economic development-particularly in 
high technology areas-have become 
increasingly visible in the past decade. 
Both trends have focused the attention 
of officials and the wider public more 
explicitly on higher education's overall 
return on investment. These concerns 
also have spawned a series of national 
reports. 

Prominently among them is Transforming 
the State Role in Undergraduate 
Education: Time for a Different View 
(issued by the Education Commission of 
the States) and Time for Results (a 
report of the Task Force on College 
Quality of the National Governors 
Association). Like the "academic 
improvement" reports that preceded 
them, these documents called for more 
and better data about the results of 
undergraduate study. In addition, 
however, they recommended that such 
information: 

(1) be publicly available and 
comparable across institutions, 

(2) be used to inform policy and 
resource allocation decisions at 
the state level, and 

(3) be appropriate to inform 
"consumer choice" on the part of 
students and their parents in the 
decision of which college to 
attend. 
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Together, these developments have 
helped to stimulate varied "assessment" 
Initiatives at both the state and 
institutional levels. Because of the sheer 
range of aitematives, some specification 
of the types of programs currently being 
undertal<en is an important prerequisite 
for a discussion of costs and 
benefits-both of which will vary widely 
depending on program scope, intent, 
location, and methodology. 

1. State Appr^oaches 
to Assessment: 

Currently, some fifteen states have 
established identifiable "assessment 
programs" through Board resolution, 
executive directive, or statute (Boyer, 
Ewell, Rnney and Mingle 1987). 
Initiatives are expected or planned in an 
additional fifteen to twenty states over 
the next five years. State approaches to 
assessment differ widely, and include 
one or more of the following activities 
(Boyer and Rnney 1987): 

o common statewide testing of the 
basic skills of incoming college 
freshmen (in place in NJ, TX, and 
TN; contemplated in WA). 

o common statewide testing of 
college students for purposes of 
promotion (FL, GA), to determine 
undergraduate program effective- 
ness (NJ, TN; discontinued in SD 
after 3 years), and/or to allocate 
resources to institutions (TN). 

A requirement that each institution 
submit and receive approval for a local 
"assessment plan" to collect information 
on undergraduate outcomes consistent 
with institutional mission (in place in AZ, 
CO, NJ. VA, SD, MO, HI, GA, Wl, SO); in 
most cases, plan approval is in some 
way linked to institutional funding. 



States have also exhibited widely varying 
levels of investment in such programs. In 
many cases, no new dollars are 
associated with the mandate. In others, 
categorical grant programs have been 
used to support "demonstration projects" 
at individual institutions. In a few cases, 
substantial new dollars have been 
invested at the state level (in NJ. 
$850.000/year for college outcomes. 
$500.000/year for basic skills assess- 
ment; in VA. institutional grants of 
$10-12/FTE student at all public 
institutions). 

2. Institutional Approaches 
to Assessment: 

Partly in response to emerging state 
mandates-most of which assign res- 
ponsibility for designing and carrying out 
assessment and evaluation activities to 
individual campuses~the past three 
years have also seen development of a 
considerable range of institutionally- 
based assessment efforts. 

Before 1984. the number of institutions 
possessing identifiable "assessment 
programs" was extremely small. Best 
estimates by 1988 indicate that some 
12-15% of American colleges and 
universities have actually established 
such programs-most of them in the past 
two years (for example. Hyman et al. 
1988; El-Khawas 1987). Additional 
national survey data (for example. 
El-Khawas 1986) indicates widespread 
attention to the ibsue. and substantial 
expectations that assessment programs 
will be implemented in some form on a 
majority of public campuses. 

In establishing the range (and cost) of 
such initiatives, it is important to stress 
that many existing institutional 
data-collection activities may be 
included. Prior to the emergence of 
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"assessmenf as an issue, most colleges 
and universities collected considerable 
evaluative informatior; about students 
througli a variety of meclianisms (Ewell 
1983). Most common are diagnostic 
testing to determine weaknesses in basic 
skills, and occasional opinion surveys 
designed to determine student 
satisfaction with campus services. Most 
institutions also collect a range of 
demographic and course perfomiance 
Infonnation about their students, 
embodied in computerized student 
record systems. Generally lacking, 
however, and explicitly addressed by an 
identifiable institutional assessment 
program, are 

(a) information on the effectiveness of 
the cuniculum as a whole as 
determined by changes in overall 
student knowledge and skill levels, 
and 

(b) an institutional mechanism for 
integrating scattered information 
on institutional effectiveness and 
using this information systemati- 
cally to improve instruction. 

While details of emerging institutional 
assessment programs vary widely, most 
contain sufficient common features to 
allow meaningful comparison and to 
support limited generalizations about 
costs and benefits. Partly this is because 
most such programs have been 
modelled on two or three widely quoted 
examples-most prominently for public 
institutions. Northeast Missouri State 
University and the University of 
Tennessee, Knoxville. Partly this 
uniformity is also due to the fact that 
most recently established programs were 
developed in response to state 
guidelines-most notably in Virginia, 
Colorado, and Missouri. Among the 
common emerging features of 



institutional assessment programs are 
the following (Ewell 1987): 

o An identified office or individual 
assigned explicit responsibility for 
coordinating the assessment 
function, for advising individual 
academic units in designing 
assessment and evaluation 
procedures, and for helping to 
interpret obtained results for 
decisionmaking. 

Generally this office or individual is given 
an identifiable budget for the activity~a 
budget that often reflects reallocation of 
existing information functions as well as 
funding new activities. 

Reporting lines vary considerably, but 
generally locate the function in close 
proximity to the institution's chief 
academic officer. 

While recruitment of an evaluation/ 
assessment specialist to manage this 
function is occurring with increasing 
frequency, most common is 
reassignment of a faculty member in 
education or social science to manage 
the activity. 

COMMON COMPONENTS OF 
ASSESSMENT PROGRAMS 

0 Basic skills testing in reading, 
writing, and computation for 
entering freshmen. 

Generally this is an existing activity, 
reassigned to the assessment function. 

At the same time, changes in this 
function are often made to render it 
more systematic. These include, 

(a) ensuring that all entering students 
are tested (often they are not, as 
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they can "place out" of such 
testing by high school record or 
SAT/ACT perfomiance-neither of 
which provide a good guarantee 
that students in fact possess the 
required skills); 

(b) use of common test instmments 
(often a range of instruments are 
used for different types of students 
with noncomparable characteristics 
and results); 

(c) mandatory placement of students 
into remediation when deficiencies 
are identified (often students are 
"advised" to take certain remedial 
courses but in fact do not do so); 
and, 

(d) retesting students as a condition 
for exiting remediation using the 
same tests or instruments that 
resulted in the initial determination 
of deficiency (often students so 
tested after "completing" 
remediation still cannot perform 
effectively enough to do college 
level work). Such changes will 
generally considerably alter the 
character and effectiveness of 
basic skills testing programs. 

o Evaluation of the effectiveness of 
"general education," by looking at 
the development in students of 
such integrative skills as critical 
thinking and problem solving. 

For most institutions, this is an entirely 
new activity, as the only existing 
information will be student course grades 
in general education courses. Common 
practices include, 

(a) administration of one or more 
standardized examinations to 
samples of students at the end of 



the sophomore or senior years to 
determine growth, 

(b) design and implementation of 
locally-designed examinations and 
exercises that require students to 
use knowledge in an integrated 
fashion to solve problems, and 

(c) special surveys and course 
evaluations that tap student 
perceptions about the effective- 
ness of general education. 

0 Evaluation of the effectiveness of 
instruction in the student's major 
field. 

The degree to which this is. a new 
activity depends upon the field of study. 
In most professional or occupational 
fields, existing certification, licensure, or 
accreditation procedures demand such 
assessment. In most traditional 
academic disciplines, no such 
procedures are typically in place. 
Consequently, establishing an explicit 
assessment program in this area 
generally entails, 

(a) ensuring that all fields undertake a 
oeriodic examination of this type 
(usually every five to seven years), 

(b) ensuring that most of the 
knowledge and skills objectives 
that the program claims to 
develop are in fact assessed in 
some way, and 

(c) determining the success of 
graduates in employment and 
further study. Often these activities 
are made a part of existing 
program evaluation processes. 

Generally they require departments and 
programs to work individually, with 
limited funding and technical assistance 
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provided through the institution's 
assessment program. 

o Determination of student 
satisfaction and behavior through 
periodic surveys. 

Depending upon past institutional 
practice, this can imply a reassignment 
or a wholly new activity. Generally 
responses are sought from both current 
and former students, and involve, 

(a) goals and goal fulfillment through 
attendance, 

(b) reactions to and satisfactions with 
various aspects of the instruction 
and services provided, 

(c) self-reported development in 
identified areas of knowledge and 
skill, and for graduates and former 
students 

(d) current activities and an evaluation 
of the strengths/weaknesses of 
instruction received. 

Often such surveys are centrally 
administered by assessment offices. In 
other cases, they are administered by 
different units across the campus, but 
are coordinated and partly supported 
through the assessment function. 

Although the emphasis placed upon 
each of these activities may vary from 
institution to institution, they together 
represent a minimum set of assessment 
functions addressed by such programs. 

All five, for example, are explicitly 
addressed by assessment planning 
documents at Arizona's three public 
universities. 



SOME CAVEATS ON ESTIMATING 
THE COSTS AND BENEFITS 
OF ASSESSMENT 

In attempting a comprehensive review of 
the costs and benefits of institutional 
assessment programs, a number of 
initial observations and caveats are in 
ord(3r. Each places important limits on 
what can mep.ningfully be accomplished. 
First, comprehensive programmatic 
assessment efforts are quite new. Fewer 
than half a dozen institutions have 
established such programs for a 
suffident length of time for their 
consequences to become clear. The 
vast majority only initiated such activity 
within the last two years and are still in a 
planning, shakedown, or implementation 
mode. ConsequenJy, their cost and 
payoff structures will probably be 
different from those typical of fully 
established programs. Despite 
similarities in coverage and function, 
moreover, institutional assessment 
programs differ markedly in adminis- 
trative organization. This means that they 
may also differ significantly in the way 
they account for invested resources- 
-particularly those having to do with 
reassigned personnel. 

Second, because establishment of an 
explicit assessment function often 
involves a reassignment of existing 
activities, actually counting invested 
resources can be difficult. What is of 
interest is less the total cost of 
"assessment activity" than the net 
marginal investment in new assessment 
mechanisms established, plus the 
resource implications of reassigning old 
functions such as placement testing and 
student surveys. Moreover, because 
such existing runctions are often 
scattered and inefficient, establishing a 
centralized coordinating mechanism can 
yield economies. As a result, past 
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attempts to estimate the cost of 
institutional assessment programs have 
concentrated on establishing the 
marginal costs of such programs, rather 
than the total investment required (for 
example Ewell and Jones 1986, L.ewis 
and Wasescha 1987). 

Third, while the claimed benefits of 
institutional assessment are many, 
consistent valuation of tiiese benefits is 
difficult. This situation is share;' by most 
analyses of higher education's benefits: 
not only are dollar equivalents hard to 
come by, but estimates of benefit will 
vary considerably across different 
individuals with differing value stitictures 
and preferences (Bowen 1980). As a 
result, there have been few definitive 
studies of concrete rate-of-retum for 
investments in instructional processes in 
colleges and universities tieyond such 
relatively well-defined technical areas 
such computer-assisted instiuction. Tliis 
situation considerably limits the 
applicability to assessment activities of 
formal "cost/benefit" techniques that 
compare the dollar values of investments 
with the dollar values of obtained results 
(Levin 1983). 

Given tiiis condition, past studies of 
educational impact have generally 
concertrated on one of two alternatives. 
Cost/effectiveness analyses examine 
invested costs in the light of some 
identifiable common output or criterion 
measure for the activity in question. But 
with the exception of degree program 
completion, few common criterion 
measures exist tor the outputs of higher 
education. Indeed, it is the very absence 
of such measures which is in part 
inspiring the call for outcomes 
assessment in the first place. The 
second alternative is to compare costs 
with "utilities""identifiable benefits which 
may vary in value across beneficiaries 



depending upon their individual goals 
and preferences. Here the primary intent 
of analysis K to determine which 
altematives provide which kinds of 
benefits to which beneficiaries at what 
cost. While such analyses do not provide 
unambiguous "cost/effectiveness ratios" 
upon which to decide what to do, they 
can nevertiieless provide considerable 
guidance for policy. 

Fourth, like benefits, costs may also be 
subject to inconsistencies in valuation. In 
addition to identifiable fiscal and physical 
resources, the establishment of a new 
and controversial program may cost 
"political capital" or may cause 
considerable organizational friction over 
and above its observable cost. Focusing 
institutional and faculty attention on 
assessment may preclude olher 
opportunities or may inhibit insti-uctional 
innovation or risk-taking. Like 
educational benefits, these effects are 
sufficiently ambiguous that converting 
them into estimated dollar equivalents 
may lose information. Moreover, beyond 
the issue of how much is invested there 
is that of who pays. Valuation of costs 
may vary significantiy depending upon 
what particular parties-at-interest actually 
have at stake in the process. 

Finally, the nature of assessment as a 
"management information" activity means 
that it hi a special kind of 
investment-one not easily subject to the 
traditional rubrics of cost/benefit or 
cost/effectiveness analysis. L'ke all 
activities related to management, 
operating an assessment program 
involves the investment cf real 
resources. These entail direct and 
observable costs to the institution. 

Benefits of information, however, are 
generally highly indirect. Literature on the 
"econorrics of information", for example. 
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is founded on management's use of 
infomiation to make effective choices 
among posed aitematives-each of which 
has an associated stream of costs and 
benefits (for example, Huber 1980, Day 
1978). If additional infcmation can 
increase the probability of choosing a 
•profitable" altemative, investment in 
obtaining tiiis information is justified up 
to tile point where the margin of 
incr' nental "profit" disappears. But 
decisionmakers in higher education are 
rarely presented witti such choices. Far 
more common is a situation where 
information-obtained at an invested 
cost-indicates that a particular 
deficiency is present and suggests some 
lines of attack toward addressing it. To 
address the deficiency, however, 
requires tiie investment of yet another 
increment of resources. Absent this 
second increment (or absent its being 
effective), investment in information may 
pay no dividends whatever. This is one 
re?5on why proponents of educational 
assessment and evaluation pay so much 
attention to explicit mechanisms for 
connecting evaluation information to 
concrete decision processes. 

Taken together, these caveats suggest a 
ti-eatinent of the costs and benefits of 
institijtional assessment activities with the 
following properties: 

0 Costs are appropriately treated as 
programmatic costs--that is, tiie 
costs associated with a formal, 
established, identifiable activity with 
sufficient organizational and 
budgetary identity that resources 
can be consistently accounted. 

o Costs are appropriately treated as 
marginal costs. 

That is the difference in investment 
between establishing an assessment 



program of given content and structure 
over existing information-gathering and 
utilization costs previously being 
incurred by the institution. 

0 Costs are appropriately treated as 
the costs of assessment itself, not 
the costs associated with actually 
addressing identified deficiencies. 

Moreover, the costs of assesment are 
part of any total rate of return calculation 
for the costs and benefits of 
instructional improvement in 
undergraduate education. 

0 Some "costs," such as loss of 
access and reduction of 
risk-taking, are more appropriately 
treated as externalities than as 
direct dollar costs; 

While it is in principle possible to obtain 
rough dollar equivalents for these costs, 
analysis is more meaningful if they s/e 
considered directly. 

0 A few claimed benefits of 
assessment, while difficult to 
render consistently in dollar terms, 
are sufficiently comparable across 
situations to in principle support a 
cost/effectiveness estimate; 

Where results are comparable, for 
example claimed increases in student 
persistence and graduation rates, 
cost/effectiveness calculations are 
appropriate. 

0 Most benefits of assessment are 
appropriately treated as "utilities" 
to particular constituer;ts in the 
process, each of which may, 

(a) value obtained outcomes 
differently, and 
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(b) pay different shares in the total 
investment. 

TTiese oroperties define the limits for the 
discussion that follows. 

PARTIES-AT-INTEREST 

Benefits of assessment have been 
claimed by proponents in bewildering 
variety. These range from improvements 
in specific cognitive outcomes for 
individual leamer; to broad societal 
benefits achieved by greater 
accountability to the taxpaying public. 
Similariy, "costs" of assessment have 
been raised by critics that encompass 
losses of opportunity for individual 
students to politicization of higher 
education that distracts the enterprise 
from its fundamental ends. In order to 
make sense, of these claims in 
determining the wisdom of embarking on 
various forms of assessment, an 
analytical framework encompassing the 
majority of these claims is required. 

Figure 1 provides the foundation for 
such a framewori< by arraying benefits 
and costs against a range of 
constituencies or "parties-at-interest." 

Benefits are roughly presented in terms 
of the degree to which they provide 
direct or indirect payoffs to the identified 
party. Costs are noted as either direct 
dollar consequences or as "externalities" 
for which calculation of dollar equivalents 
are for the most part inappropriate. 
Parties-at-interest in this scheme are 
identified as follows: 

o Individual students 

are individuals directly involved In higher 
education as "consumers", including 
currently enrolled students, former 
students, and potential students. 



0 Faculty 

are members of higher education 
faculties currently or potentially involved 
in providing undergraduate instruction. 

0 Institutions/Programs 

are colleges and universities as 
organizational entities, or subunits within 
institutions that can function relatively 
autonomously. 

0 External Constituents 

are organizations external to higher 
education that employ the graduates of 
colleges and unix'ersities and that stand 
to gain or lose because of skills or 
deficiencies possessed by graduates- 
-either in general or in particular 
programs (these may include 
business/industrial employers, local 
governments, social service agencies, 
etc.) 

0 The Public 

are members of the general public 
(either as taxpayers or as the 
beneficiaries of improvements in general 
public welfare due to an educated 
public) and their elected representatives. 

Each of these parties-at-interest may 
benefit or bear costs in different 
proportion depending upon the types of 
programs proposed. 

This framework is intended to serve two 
distinct analytical purposes. 

First, it can serve as a guide for 
systematically summarizing existing 
arguments and evidence about the 
actual impacts of institutional 
assessment programs. 
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Reviewing this body of material cell by 
cell for different parties-at-interest will 
constitute the balance of this section. 

Second, the framework can be used to 
compare the benefits and costs of 
different proposed alternatives to 
particular constituendes-fbr example a 
common state-mandated baisic skills 
placement examination aga'nst the 
establishment of incentives for 
institutions to create their own programs. 

While such an analysis does not provide 
a definitive judgment about relative 
program cost/effectiveness, it will at 
least enable policymakers to select the 
most promising alternatives. Several 
such programmatic trade-offs will be 
considered in a following section. 

1. Benefits of Assessment by 
Identified Parties-at-interest: 

Benefits of assessment have been 
claimed and documented in a variety of 
areas. Evidence for each will be briefly 
considered by potential beneilciary 
groups as outlined in Figure 1. 

a. Individual Students: 

Among the claimed benefits of 
establishing assessment programs for 
individual students are the following: 

0 Documentable increases in 
knowledge and skills. 

Direct increases in student knowledge 
and skills are held to be a product of 
both the individual feedback on 
performance provided by testing and of 
improved ability to target teaching on 
identified weaknesses in student abilities. 

Evidence from bot.) state and 
institutional testing programs is 



fragmentary, but indicates that modest 
gains are partiaiiy attributable to 
assessment processes. In most cases, 
however, t^lese gains are confined to 
so-called "basic skills" areas-reading, 
writing and computation. In Florida, for 
example, demonstrable increases in 
student writing ability have been 
associated with the state's College Level 
Academic Skills Prograni (CLASP) which 
involves testing sophomores in basic 
skills as a condition for advancement 
(Ciereszko 1987). At institutions such as 
Northeast Missouri State University 
where repeated administration ot a test 
instrument can document the "value 
added" of instruction, percentage gains 
in English subscores as measured by 
the ACT Assessment Examination rose 
from 4.8% in 1978 to 10.6% in 1983; this 
period saw heavy investment by the 
university in enhancing assessment 
processes and in using the results to 
improve curriculum (McClain and 
Krueger 1985). In both cases, benefits 
are somewhat offset by the fact that the 
test-taking populations became more 
able due to selection biases: in Florida, 
less able students were deterred from 
taking the examination while at Northeast 
Missouri admissions standards rose in 
the same period. Similarly, it was not 
assessment alone that caused any 
increases but rather the combination of 
assessment and appropriate curricular 
action. If used to inform curricular 
intervention, however, modest gains in 
overall levels of basic skills can generally 
be expected. 

0 Increases in persistence and 
graduation rates due to improved 
advisement, placement and 
retention efforts. 

Student assessment programs have also 
been widely associated with gains in 
student persistence and graduation. This 
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is due primarily to two different 
mechanisms. 

First, at the level of individual courses, 
coupling assessment of pre-requisite 
skills with course registration policies can 
result in significant increases in success 
rates. At North Carolina State University, 
for example, directed placement into 
mathematics and pre-engineering 
courses resulted in 30-40% increases in 
course passage rates overall, and over 
70% increases for minority students 
(Ewell 1984b). 

Second, institution-level assessment 
results-particularly the results of student 
attitudinal and perception surveys-can 
result in documentable increases in 
overall rates of student retention and 
degree completion. At Northeast 
Missouri, for example, five-year degree 
completion rates have risen 
approximately 15% since initiation of 
comprehensive assessment (Northeast 
Missouri State University 1987). More 
typical, however, are the experiences of 
such institutions as Towson State 
University (MD) and other members of 
two national student outcomes 
assessment projects in the early 1980's 
that increased retention rates by 8-10% 
overall in three years (Ewell 1984b; 
Kemerer, Baldridge and Green 1982). 
Increased retention rates, of course, are 
of individual benefit in raising a given 
studerit's overall chances of obtaining a 
degiee, and therefore of receiving the 
payoffs associated with degree 
completion. 

o Increased credibility of the degree 
obtained. 

Another claimed benefit of assessment is 
public demonstration of the actual 
"learning content" of a baccalaureate 
degree (McClain 1984). Given 



considerable evidence of wide variations 
in student ability among the products of 
different institutions, one claimed 
objective of student assessment is to 
provide concrete evidence of what 
students know and can do upon 
graduation. The claimed individual payoff 
in this case is increased competitiveness 
of graduates in the employment market. 

No concrete evidence cjrrently exists to 
support this proposition beyond the 
generally better placement rates 
experienced by graduates of some 
institutions participating in assessment. 
Much of this success, however, could 
equally be attributed to the fact that 
these institutions have simultaneously 
raised their admissions standards. 

o Improved information for making 
initial college choices. 

Rather than affecting currently enrolled 
students, these benefits are held to 
accrue to potential college students and 
their parents. Publicly available 
information on attainment levels, success 
rates, and the placement experiences of 
graduates may enable more informed 
consumer choices about which college 
or university to attend. The perceived 
success of graduates has traditionally 
been a major drawing card for "selective" 
institutions (Zemsky and Oedel 1983). 

Evidence that assessment results can to 
some extent offset entering student 
ability in the perceived attractiveness of 
an institution is provided by the 
increased application pools at such 
institutions as Alverno College, Kings 
College, and Northeast Missouri where 
assessment programs have been in 
place for p number of years. As above, 
however, these increases have been 
accompanied by highly visible 
innovations in curriculum in addition to 



707| 2 



assessment, 
h. Faculty: 

At the level of individual faculty 
members, few concrete benefit are 
claimed through assessment. Two, 
however, are of sufficient prominence to 
note below: 

o Increases in the ability to target 
teaching on the identified needs of 
members of a giver> classroom. 

Faculty members at institutions such as 
Kean College (NJ) and Kings College 
(PA) report that pre-assessments of 
student knowledge and ability 
significantly improve their ability to tailor 
instructional material to appropriate 
levels (for example. Farmer 1988). This 
allows important deficiencies in 
prerequisite knowledge to be quickly 
identified and dealt with before the term 
is too far advanced. It also allows 
adjustment in the timing and coverage of 
class content at an early point in the 
term. 

o Increases in faculty ability to 
design and administer classroom 
tests. 

While the basic intent of comprehensive 
outcomes assessment programs is to 
examine performance across individual 
classes and students, significant 
numbers of faculty members a all types 
in institutions report that the engagement 
in the design of assessment has 
improved their ability to design better 
tests for their own use (for example, 
Calhoun 1986; Banta 1985; Banta and 
Moffett 1987). This experience has been 
a particularly apparent effect of 
Implementing assessment in the major 
field or department at research 
universities. Similar impacts have been 



reported with less frequency in general 
education at comprehensive colleges 
and liberal arts colleges (Curry and 
Hager 1987; Farmer 1988). The key 
here, however, is that faculty must have 
beer^ actively involved in designing and 
administering the assessment program, 
though there is some evidence that 
participating in commercial tests such as 
the ACT-COMP can also result in 
modest faculty development benefits 
(Banta 1986; Forrest 1982). 

c. Institutions/Programs: 

By far the greatest claims of positive 
benefit for systemat.c assessment have 
been at the institutional or program level. 

Among them are the following: 

o Improvements in the sequencing 
and structL d of undergraduate 
curricula. 

Most institutions engaging in 
comprohsnsive assessment report 
significant early curricular changes as a 
result. Many of these changes, it is 
interesting to note, appear to have 
occurred as a result of designing the 
assessment (Banta 1986; bwell 1984b). 

In several reported cases, curricular 
change pre- dated actual data collection. 

Documented curricular changes resulting 
from assessment activities have been of 
three distinct kinds. 

First, there have been notable changes 
in the sequence of particular courses 
making up particular curricula. For 
example, at Mt. Hcod Community 
College, course order in several 
technology programs was reordered- 
-yielding increases of 10-12% in degree 
completion rates (Ewell 1985); at the 
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University of Tennessee, Knoxville, 
introductory courses in several major 
fields were made mandatory (Banta and 
Moffett 1987). 

Second, assessment programs have in 
several cases led to identification and 
elimination of "soft-track" courses: at 
Nortfieast Missouri, elimination of such 
courses in mathematics and requiring 
students to take more solid math 
courses resulted in visible gains in 
sophomore-level math test scores 
(McCla'n and Krueger 1985). 

Finally, assessment results have led to 
the introduction of new couises to 
address obvious cunicular deficiencies: 
a good example is provided by the 
University of Tennessee, Knoxville where 
surveys of cun-ent and former students 
in Business indicated the need for 
courses in Business Law (Banta and 
Moffett 1987). In each of these cases, of 
course, the demonstrated link is between 
assessment and changes in the 
cuniculum-changes that are presumed 
to benefit students in the form of 
increases in knowledge and skills. No 
direct evidence of such increases have 
as yet been demonstrated. 

0 Improved institutional planning and 
resource allocation. 

A second set of institutional benefitr, 
concerns the reported utility o' 
assessment results in making better 
administrative decisions-particularly 
those involving program continuation, 
investment, and disinvestment. For these 
benefits to occur there must be a strong 
linkage between assessment information 
and a regular and systematic process of 
{.rogram evaluation or review. At 
institutions where such linkages are 
present-for example the State University 
of New york at Albany-both institutional 



administrators and program/department 
chairs report that better decisions are 
made about resource allocation (Ewell 
1984b). Consistent with overall mission 
and the results of strategic planning, 
investments are targeted toward 
identifiecl defidencies in cuniculum in 
part discovered through assessment 
activities. Similar experiences have been 
reported by top administrators at the 
University of Tennessee, Knoxville, 
where assessment results are also visibly 
used in the program review process 
(Banta and Fisher 1984). in some 
reported cases, additional assessment 
information has determined a program 
continuation decision which would 
otherwise have been different. At St. 
Petersburg Junior College, for example, 
infonnation about post-grp'' ite success 
and favorable employer latings for 
graduates changed a Board decision 
about continuing an "inefficient" program 
in Social Work (Ewell 1985). 

o Increases in faculty time 
committed to teaching. 

A number of observers have reported 
gains in faculty "time on task" expended 
in support of undergraduate instruction 
as a result of planning and implementing 
comprehensive assessment activities. 
Such reports have been particularly 
frequent with regard to general 
education-an area which often claims 
last priority in allocations of faculty time 
and attention. In such cases, faculty 
report talking more with their colleagues 
about general education issues, 
discussing particular course content with 
greater frequency, and spending more 
time preparing for classes and talking 
with students. Concrete evidence of this 
effect is scanty, however, and relies only 
on unsystematic self-reports from faculty 
and departmental administrators. 
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0 Enhanced ability for Institutions to 
obtain additional resources. 

Heightened Interest in assessment per 
se h«is, of course, enhanced the 
wjlliny>iess of both public and private 
funders to provide resources to directly 
support such activities. Those institutions 
that have instituted programs early have 
generally been able to gamer such 
funding-often in substantial amounts. 
More signlficarttly, institutions tiiat have 
collected assessment results over time 
have been able to use this information 
to make a better case for budgetary 
requests from state governments and for 
spec'al program requests from 
foundations and other funders. In 
Missouri, for example. Northeast 
Missouri has consistently supported 
special appropriation requests from the 
legislature witti a range of concrete 
assessment information-most recently in 
a library enhancement request (Krueger 
and Heisserer 1987). Interviews with 
state legislators in Missouri indicate that 
the availability of such information was a 
significant contributor to the decision to 
fund the institution's request-both 
because it directly supported action and 
because it provided indirect evidence of 
good management practice (Ewell and 
Boyer 1988). Similarly, Kings College 
(PA) has used its assessment program 
to obtain successive curriculum 
improvement grants from several 
national and local foundations (Farmer 
1988). 

d. External Constituents of Higher 
Education: 

A number of benefits of assessment aro 
claimed for agencies and institutions that 
employ higher education's graduates. 
Few are to date substantiated by any 
direct evidence. They include the 
following: 



o Increased assurance that potential 
employees possess requisite basic 
and higher order skills. 

Many institutions that err.ploy college 
and university graduates have over the 
years been forced to design and 
implement their own testing programs to 
determine the reading, writing, and 
computational skills of potential 
employees. Largely they report 
implementing such programs because 
possession of a baccalaureate degree 
can no longer be taken as sufficient 
evidence that individuals possess 
required skills. Recentiy, for example, the 
US military ceased to accept a college 
degree as the sole source of evidence 
about basic skills and is requiring special 
testing to determine skill levels among 
officer candidates (Ewell ig84a). 
Credible higher education assessment 
would presumably obviate the need for 
such special testing programs, resulting 
in significant cost savings for employers. 

o Increased assurance that potential 
employees who are graduates of 
specific job-related programs 
possess the special skills 
necessary to perform effectively. 

This benefit parallels the above, but with 
respect to specific, identifiable, 
job^related skills. Institutions that have 
instituted end-of-program comprehensive 
examina.1ons, practice, or projects in 
which job-related skills ' can be 
demonstrated in a real or simulated job 
setting have reported that potential 
employers are more likely to hire their 
graduates. Results of this kind have 
been particularly apparent in Nursing 
and other health professional training 
programs. As above, demonstration of 
skill before employment obviates the 
need for employers to "remediate" 
deficiencies after employment, again 
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resulting in a potential cost saving. 

o Increased assuranca that potential 
employees possess appropriate 
attitudes toward work. 

Increasingly, employers report that they 
are concerned about whether otherwise 
well-trained college and university 
graduates possess important lob-related 
attitudes such as motivation, persiblence, 
and interpersonal values that support 
teamwork. These concerns have been 
instrumental in the efforts of some 
professional program areas to explicitly 
stress the development and assessment 
of such attitudes while enrolltKj-for 
example the American Association of 
Collegiate Schools of Business. To the 
extent that these efforts are successful, 
they are presumed to represent a payoff 
to employers in the fonri of a more 
productive workforce (e.g., Bowen 
1977). 

e. The General Public: 

Claimed benefits of assessment for the 
public at large rest largely on arguments 
of improved accountability. 

Specifically, assessment pro^'arrs are 
held to be related to the foUowir 

o Increased assurancii' 'U'^- public 
tax dollars allocated ro higher 
education are appropriately 
invested. 

Much of the argument for assessment 
from public officials rests on the 
proposition that the publ>c has a right to 
know the effects of invested resources. 

Absent specific performance information, 
public accountability at least demands 
evidenct' of effective and prudent 
management of public enterprises. 



Assessment programs are argued on 
both grounds. Neither is in any way 
quantified or quantifiable. 

o Increases in general public welfare 
associated with other identified 
benefits. 

Many claims have historically been made 
about the general benefits of an 
educated citizenry. They include overall 
increases in social productivity, cultural 
and personal benefits, and avoidance of 
such negative social consequences as 
crime and social disruption (Bowen 
1977). The majority of such benefits can 
be seen as "secondary effects" of one or 
more identified benefits above. 

2. Dollar Costs of Assessment: 

Despite the fact that assessment is 
relatively new as a distinct activity, its 
direct dollar costs can be more 
reasonably estimated than its claimed 
benefits. Partly this is because many 
associated costs are incurred early, and 
can consequently be observed more 
readily. Partly it is because the basic 
ingredients of direct cost required to 
operate assessment programs are 
comparatively well defined. As a result, 
several past attempts have been made 
to establish the direct costs of such 
programs (Ewell and Jones 1986; 
Conrad and Wilson 1985; Bowen 1985; 
Lewis and Wasescha 1987). It is 
important to note that these studies have 
themselves been used by several state 
governments (for example, Virginia and 
New Jersey) to establish guidelines for 
funding institutional assessment 
programs. Consequently, actual available 
direct cost data has tended to converge 
in recent years: following Bowen's 
analysis of expenditures in higher 
education (1980), the amount of fundin 
provided itself tends to determine 



"costs", because institutions spend all of 
what they are given. Assessment 
programs are no less subject to this 
observation than are any of the other 
things that institutions do. 

Any discussion of assessment costs 
involves two distinct topics. 

First, it is necessary to actually estimate 
the direct costs incurred-based on both 
the actual cost components of an 
assessment program and on an analysis 
of what institutions actually spend. 

Second, it necessary to determine who 
pays these costs. 

a. Cost Estimates for Assessment: 

Two methods are generally used for 
arriving at estimates of program costs. 
The most frequent is "component 
costing" in which an estimate of total 
program cost is constructed by 
determining the costs associated with 
each of the program's requisite 
elements. A second method~"statistical 
costing"--involves analyzing a number of 
existing programs and calculating what is 
actually spent on program activities. 
Both methods have been used to 
estimate the operating costs of 
institutional assessment programs. Due 
to small numbers of existing programs, 
however, statistical methods have 
generally been used only to confirm the 
estimates derived from component 
costing. 

In implementing assessment programs, 
institutions incur costs in four areas. 
They include the following: 

0 Instrument costs. 

Various cognitive tests and student 
surveys constitute the technical basis for 



any assessment data gathering effort. 
Before they can be fielded, such 
insUuments must be developed or 
obtained, if they are developed locally, 
costs are incurred by faculty and 
measurement specialists. After initial 
development, such instruments can be 
produced on a regular basis-generally 
at lower cost than comparable 
commercial instruments. Institutions that 
have developed their own examinations 
have usually treated their construction as 
a departmental activity. Common 
practice, for example, is to provide each 
department with grants of $2000-3000 to 
cover the direct costs of test 
construction. This practice tends to bury 
many test-making expenses in ongoing 
departmental administrative budgets. If 
the full cost of such activity were 
calculated, it would undoubtedly be 
higher than budgeted amounts. 
Commercial tests and surveys are 
usually purchased on a per unit basis. 
Costs range from a low of $7/instrument 
for such widely administered tests as the 
ACTAssessmentto$75-$100/instrument 
for specialized field examinations. Costs 
for commercial cognitive instruments 
generally include analysis and scoring as 
well as acquisition of the instrument 
itself. 

Commercial surveys are also generally 
purchased on a per unit basis, with 
costs ranging from $0.25/survey to 
approximately $1 /survey. When analysis 
services are also used, these costs 
average $3-$5/ survey. In most cases, 
minimum orders or service charges of 
approximately $100 apply. 

0 Administration costs. 

Once acquired, tests and surveys must 
be administered to students. At some 
institutions, existing testing centers 
established for placement or diagnostic 



ERIC 



17 



712 



testing may bear some of this burden. In 
most cases, however, the number of 
instruments to be administered will 
require resources beyond those available 
in existinjg testing centers. At minimum, 
test administration requires personnel to 
serve as proctors. Md(|^ver, special 
equipment such as projlMbrs and tape 
recorders may be required for some 
types of examinations (for axample, the 
ACT-COMP). Survey questionnaires will 
either be aidministered in dass or by 
mail. In the first case, direct costs of 
administiBtion will be minimal-though 
some personnel costs will be incurred. In 
the second case, experience indicates 
incurred costs of $2-$3/completed 
instiument should be budgeted-an 
estimate tiiat includes more than one 
follow-up mailing, possible telephone 
fbllow-up, and tiie costs of monitoring 
ttie follow-up process. 

0 Analysis costs. 

As noted atsove, commercial cognitive 
tests generally include analysis and. 
processing expenses in ttie price of the 
instilment. Scoring and analytical 
services ttiat cover ttie costs of data 
enti7, computer analysis, and production 
of a simple summary report are also 
available for most commercial surveys. 
To be of maximum use, however, 
obtained data will probably be analyzed 
locally, requiring additional personnel 
and data processing costs. This will be 
particularly tine for locally-designed 
assessments, where data coding 
schemes and analyses must be 
designed from scratch. In the initial 
stages of such a process, considerable 
care must be taken to develop 
error-checl<ing procedures and methods 
for handling incomplete or contradictory 
information. Once such procedures are 
put in place, however, ongoing costs for 
analysis will be minimal-principally 



involving personnel and computer time. 

o Coordination costs. 

Most institijtional assessment programs 
involve establishment of a central 
function to coordinate a range of data 
collection activities and to provide 
technical assistance to departments and 
units in designing, implementing, and 
interpreting the results of local 
assessments. Costs associated witii 
establishing an office or function of this 
kind include ttiose for new professional 
and support staff, space to house these 
personnel, and ongoing operating 
expenses. Established offices of this kind 
range from .5-2.0 FTE professional staff, 
depending upon institutional size, 
commitment, and the complexity of the 
program. In many cases, existing 
personnel are reassigned to provide 
staffing for an assessment center. 
Faculty with appropriate research 
backgrounds in ttie social or behavioral 
sciences, or Testing/Institutional 
Research professionals may be taken 
from their current assignments and given 
responsibility for coordinating 
institution-wide assessment activities, 
designing instruments, or for analyzing 
and interpreting test and survey results. 

In such cases, the relevant cost 
consideration is that of replacing the 
reallocated staff member In his or her 
original function. 

Based on these cost components, most 
past estimates of appropriate component 
costs for institutional assessment range 
from $7-$10/FTE student. Such 
estimates must be treated with extreme 
caution, however, as appropriate 
programs will vary greatly based upon 
institutional mission, the number and 
types of instructional programs offered, 
and characteristics of the student body. 
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EstliTtates will also vary greatly 
depending upon whether and how 
sampling strategies are used and the 
degree to which the institution has 
already developed appropriate entry 
testing and survey follow-up procedures. 

Statistical estimates of assessment costs 
based upon actual incurred costs at 
established institutions yield a somewhat 
higher estimate on a per-student basis. 

Program budgets for Northeast Missouri 
and the University of Tennessee, 
Knoxville-both of which have operated 
identifiable assessment programs for 
over five years-average a bit over 
$10/FTE student annually. Emerging 
institutional programs, for example those 
at James Madison University and Kean 
College, are cun"ently incurring costs of 
about $12-$15/FTE student each year. 
In such cases, however, higher costs are 
expected because considerable 
instrument development is being 
undertaken in the progranVs first years. 

Institutional budgets for assessment 
submitted by all public institutions in 
Virginia last year documented expected 
costs ranging from about $8/FTE to 
$35/FTE. Funds actually allocated by the 
Virginia State Council for Higher 
Education for institutional assessment 
activity for next year averaged $12/FTE 
across institutions. 

b. Who Pays? 

While the direct dollar costs of 
institutional assessment can be 
estimated with reasonable confidence 
within a range of $7-12/FTE student, the 
question of who bears these costs can 
vary considerably. Different methods of 
allocating costs among the principal 
parties-at-interest will, of course, heavily 
determine their net payoff as a result of 



the activity. Existing institutional practice 
has involved direct dollar costs incurred 
by one or more of the following: 

0 Individual students. 

Some institutions pass on the costs of 
assessment directly to individual 
students in the form of a testing fee. 
South Dakota's statewide assessment 
program, for example, is funded by a per 
student charge of $7/student, which 
covers all direct dollar costs of acquiring 
and scoring mandated instruments 
(Boyer and Ewell 1988). 

Other individual institutions levy testing 
fees ranging from $7 to $9 per student. 

Generally, the case for charging students 
is made on the basis of the direct 
benefits that students receive ft-om the 
operation of a testing or assessment 
program. 

0 Faculty. 

While individual faculty members may 
incur considerable opportunity costs and 
may invest considerable time through 
their participation in assessment, they do 
not generally bear any dollar cost. 
Indeed, if the institution chooses to 
invest heavily in the development of local 
instruments, faculty may receive direct 
dollar payments for their involvement. 

Generally, however, these are in lieu of 
teaching and do not represent an 
increment of total income. 

0 Programs/institutions. 

Where additional dollar support for 
assessment is not provided to 
institutions or programs for engaging in 
assessment-either through direct 
allocation or through a student fee 
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structure-the cost of assessment must 
be directly borne through reallocation. In 
Missouri, for example, the total cost of 
mandated institutional assessment must 
be supported by institutions themselves, 
as no new resources have been 
provided. Even in cases where some 
additional support is provided, it is often 
insufficient to cover the complete cost of 
the activity. Within institutions, for 
example, support provided to individual 
programs or departments for developing 
and piloting a msyor field examination is 
approximately $2000-$3000. Given actual 
incurred costs, this amount may cover 
only half to two-thirds of the requirement. 

o The public. 

To the extent that direct increments of 
dollar cost are not covered by student 
fees, they are ultimately incun-ed by 
taxpayers. On a per-taxpayer basis, 
however, such additional costs will be 
small compared to tota' payments for 
higher education. New Jersey's 
statewide College Outeomes Evaluation 
Project-cun-entiy the largest such 
statewide developmental effort-is 
budgeted at $850,000/year. Virginia has 
provided institutions with a total of $4.4 
million for assessment in the coming 
year. Such investments are well within 
public program evaluation guidelines that 
call for t!:e investment of 1-2% of total 
budget in evaluation activities, and well 
below typical corporate investments of 
5-8% in research and development 
activities (Bowen 1977; Lewis and 
Wasescha 1987). 

3. Some "Externalities" of 
Assessment: 

Not all identifiable negative 
consequences of assessment can be 
appropriately treated as dollar costs. 
Some represent negative individual or 



social consequences that could be 
costed out, but for analytical purposes 
are more beneficially identified 
individually. Examples include limits on 
access to higher education that 
disproportionately affect members of 
minority groups. Others represent 
demonstrated consequences of 
assessment that may be negatively 
valued by some constituents but not so 
by others, or that represent truly 
non-quantifiable results. Examples 
include narrowed choices of programs 
or courses within curricula on the part of 
students, or limits on the freedom of 
individual faculty members to teach what 
they please as they please. Each of 
these identified extemalities is treated by 
its associated party-at-interest below. 

a. Individual Students: 

Among the negative consequences 
experienced by individual students 
attributable to assessment are the 
following: 

o increased burdens associated with 
testing extornal to the classroom. 

In cases where assessment is not built 
into the curriculum and involves 
additional testing or surveys, students 
pay a cost in lost time. This cost is 
particularly apparent where assessment 
results are used solely to inform 
curricular or program decisionmaking 
and thus have no individual 
consequences for students (Banta 
1988). The extent of this cost can be 
estimated in two ways. 

First, the amount of time committed can 
be directly estimated: for most 
institutional assessment programs, time 
commitments for students will average 
three to four additional hours per year. 
This result could be costed out at 
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prevailing student wage rates to obtain 
one estimate of lost benefits to students. 

Second, some institutions have 
attempted to pay students directly fOr 
participation in assessment: quoted 
amounte range from $10-20 for a two or 
three hour testing session, generally held 
on an evening or weekend. It is 
important to note, however, that few 
students consider the reimbursement 
worth the effort. Indeed, widespread 
institutional experience with the 
difficulties of motivating students to 
commit time to assessment absent a 
direct evaluative consequence indicates 
this problem to be considerable for 
assessment programs that provide 
programmatic information only. 

0 Loss of access and choice with 
respect to desired courses or 
programs. 

Two kinds of negative consequences for 
students have been reported. 

First, if assessment results are used to 
restrict registi-ation in particular courses 
and programs, individual students may 
be barred from attending. If the primary 
consequence is to give students the 
opportunity to remediate deficiencies so 
that tiiey may subsequently successfully 
complete desired courses, this loss is 
justified-although students may suffer 
time delays in completing their 
programs. But if a consequence is to 
deny access absolutely, a significant 
cost may be borne-particularly for 
members of minority groups who may 
perform poorly on formal assessments 
for a number of historical reasons. 

Second, if results are used to inform 
curricular decisionmaking, an important 
consequence may be to change the 
structure of curricula. This may result in 
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elimination of preferred courses, or 
making more uniform the sequence in 
which particular courses are offered. 
Both limit choice on the part of individual 
students as "consumers." Both may also 
imply that some students will take longer 
to complete their coursework. In both 
respects, moreover, assessment tends 
to result in curricular changes that limit 
an individual student's "right to fail." 

0 Inappropriate limits on access due 
to embedded biases and inequities 
in assessment procedures. 

If assessment procedures are used for 
"gatekeepinQ""that is to deny individual 
students access to particular courses or 
programs, or to higher education 
generally--particular negative 
consequences may be experienced by 
members of identifiable student groups. 

Three types of students are generally 
mentioned as negatively affected. 

First, experience has shown that 
members of minority groups may be 
adversely affected by testing, due to 
both cultural biases in testing 
procedures and to historic deprivations 
in educational opportunity. Where such 
inequities are apparent, students may 
not only be directly affected but may 
also withdraw from programs because of 
anticipated negative impacts. In Florida, 
for example, the CLASP program has 
resulted in a more than 50% decline in 
degree-seekers in community college AA 
Transfer programs (Ciereszko 1987; 
Losak and Wright 1983). 

Second, assessment procedures may 
not reflect important differences in 
learning style among students, and may 
consequently not appropriately classify 
them as deficient in certain skills. 



Rnally, most assessment instruments, 
regardless of their actual content, are in 
fact predicated on a test-taker's 
language abiiity-particulariy the ability to 
read English (Adelman 1988). 

This may imply disproportionate negative 
consequences for those with low verbal 
skills but with otherwise adequate 
abilities, or fbr those whose native 
language is not English. 

b. Faculty Members: 

Negative consequences of assessment 
identified fbr individual faculty members 
are as fbllows: 

o Increased burdens involved in 
designing assessment instruments, 
administering such instruments, 
and interpreting results. 

Emerging experience indicates that 
significant amounts of faculty time are 
involved in designing appropriate 
assessment activities. Commitments are 
especially heavy when faculty-designed 
instruments are used. For institutions 
that have accounted for such time, 
estimates of this commitment 
approximate one quarter of a full-time 
faculty equivalent term teaching load to 
develop a local departmental 
examination or assessment procedure. 
Institutions such as Kean College (NJ) or 
Kings College (PA) that rely heavily on 
faculty scoring of local instruments 
report heavy time commitments in 
assessment-approximately eight 
additional hours per term to grade such 
instruments. In both cases, however, it is 
arguable that this investment is part of 
regular teaching assignments because it 
is also part of the student grading 
procedure. Indeed, at an institution like 
Alvemo College, distinctions between 
"assessmenf and "teaching" have no 



meaning; both are part of the 
established curriculum. For other 
institutions, however, assessment may 
involve an added time commitment on 
the part of faculty that is seen as 
essentially unrelated to classroom 
instruction. This is mae likely the case 
where commercial instruments are used. 

0 Perceived violations of academic 
freedom. 

Assessment involves establishing 
inherent limits on what a faculty member 
can do in his or her classroom. As noted 
above, one impact is to change 
cumculum structure and content to 
ensure greater continuity among courses 
(and particularly across different sections 
of the same course). Some faculty may 
see this as a threat to their individual 
autonomy as reflected in the concept of 
academic freedom. It is important to note 
that no legally sustanable case for 
violations of academic freedom has 
historically been heard on this bas' :, and 
that the concept refers only to a faculty 
member's right to express personal 
opinions in the classroom; no case can 
be legally made fbr a faculty member not 
teaching assigned material, or for not 
teaching effectively. Nevertheless, 
confusions over the precise meaning of 
"academic freedom" in the minds of 
some faculty often cause notable and 
important political opposition to 
assessment. 

c. Institutions/Programs: 

Negative consequences at the institution 
or program level associated with 
establishing assessment programs have 
been noted in several areas. They 
include the following: 
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o Excessive limitations on curriculum 
content and 'teaching to the test." 

One daimed benefit of assessment lies 
in its impact on curriculum-particularly in 
areas of structure, sequencing and 
uniform content. Where this is process is 
deemed excessive, it is also seen as a 
negative impact. If assessment results 
are taken too seriously in detemnining 
what should be taught, there is a fear 
that only "measurable" content will be 
included. If results have significant 
consequences for the fates of individual 
students, moreover, incentives are 
strong to tailor instruction to the 
examination rather than viewing 
performance on the examination as 
representative of a broader range of 
student knowledge and abilities. 

Evidence from Rorida indicates that the 
CLASP program has had a recognizable 
impact on the teaching of freshman 
courses in English and Math in Rorida 
public institutions. Higher pass rates on 
the examinations may be the result, but 
some faculty fear that areas not 
assessed by the examination are as a 
result being crowded out of the 
curriculum. 

o "Opportunity costs" of substantial 
investments of faculty time. 

Time spent by faculty in developing 
assessment instruments, administering 
them, and interpreting the results have 
already been noted as potential negative 
consequences for individual faculty 
members. In addition, the institution or 
program must consider the alternative 
uses of teculty time that such 
investments preclude. Generally these 
are cited in terms of lost time for 
research and public service activities. 
While no data is available to estimate the 



Impact of such diverted effort, faculty 
members at the University of Tennessee, 
Knoxville~the only public research 
institution with substantial experience 
with student assessment-have been 
reluctant to invest substantially in 
assessment programs for more than a 
three-year period. Moreover, where 
assessment is external to the classroom 
and also involves a faculty time 
commitment, less time may be available 
for actual instruction. In several 
institutions where commercial 
examinations are used, for example, one 
classroom period per term is used to 
administer the examination-a diversion 
of from 2-4% of overall instructional time. 

0 Increases in administrative 
"overhead" associated with 
creating a new function. 

If assessment is undertaken as a distinct 
and visible activity, as mandated by 
many states, it may require 
establishment of an explicit office and 
associated reporting structure. This may 
entail costs and externalities beyond 
those associated with the tangible assets 
invested in the program. Examples of 
such externalities include increases in 
administrative complexity due to 
establishment of an additional office or 
function, difficulties of coordination with 
other units assigned to collect 
information about aspects of institutional 
performance, and possible duplicative 
reporting burdens on academic units 
(Eweil 1988). While such additional costs 
can be minimized through a 
comprehensive approach to institutional 
management information and planning 
functions, emerging experience has 
shown that incurring such costs is 
likely-particularly in the early stages of 
establishing an institutional assessment 
program. 
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APPLYING THE FRAMEWORK TO 
SOME POUCY CHOICES 

As emphasized throughout, derivation of 
a single cost/benefit estimate for all 
Institutional assessment programs is 
both impossible and inappropriate. 

Difficu^i:«^s in consistently specifying 
costs and benefits in dollar terms, differ- 
ences among proposed "assessment" 
programs in different institutional 
settings, and differences in the ways key 
participants value similar outcomes all 
conspire against such a simple answer. 

What is possible is to use identifiable 
costs, benefits, and externalities to help 
sort through some proposed policy 
alternatives. Two such alternatives are 
briefly discussed in this section, using 
the proposed framework as a guide. 

1. Statewide Testing of Basic Skills: 

One leading set of alternatives centers 
on the perceived need to assess and 
remediate students entering higher 
education with respect to such basic 
skills as reading, writing, and 
computation. Statewide programs of this 
kind are currently in place in New 
Jersey, Tennessee, and Texas, and are 
being actively discussed in several other 
states. Basic features of such programs 
include, 

(a) use of a single standardized 
examination by all institutions in 
the state, 

(b) prohibitions against the use of 
results to deny admission to 
institutions, 

(c) mandated use of results to place 
students assessed as deficient into 



apprDpriateremediation programs, 
and 

(d) exit testing of students using the 
same instrument on completion of 
remediation. 

Figure 2 presents major areas of benefit 
and cost for such a program. Using 
categories of benefit, cost, and 
externality discussed in the previous 
section for each party-at-interest, the 
figure roughly indicates the presence 
and intensity of each. Particularly notable 
are the following: 

o Benefits. 

Program benefits will occur particularly 
for individual students in the form of 
documentable increases in knowledge 
and skills and in increased chances of 
graduation. But both benefits, it is 
important to note, will only occur if 
remediation and associated placement 
are also sucjessful-they are not direct 
results of the assessment program per 
se. Some benefits should also result for 
faculty, who are better enabled to teach 
at the appropriate level rather than being 
required to constantly remediate in the 
classroom. Moreover, external 
constituents and the general public will 
receive secondary benefits in the form of 
greater assurances about the skill levels 
of college studerrts and greater 
accountability. 

o Costs. 

Overall dollar costs for a testing program 
of this kind can be estimated from past 
experience in New Jersey and emerging 
experience in Texas. Both incurred initial 
test development costs in excess of one 
million dollars. Operating costs are about 
$9-11 /student tested, or about 
$4/enrolled FTE student. How such 
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costs should be paid is another matter. 
Both states currently fund the program 
out of general revenue. This entails a 
direct cost to taxpayers as well as an 
opportunity cost to institutions that might 
have received this funding for alternative 
purposes. Alternatively, direct costs 
could be passed on to students in the 
form of fees. In this case, there is a 
further choice between spreading the fee 
across all enrolled students as part of a 
tuition payment, or applying it only to 
those tested as they are tested. The 
latter altemative would have the 
advantage of clarity, but would be 
particularly burdensome for those 
assessed as deficient and forced to 
leave the institution. 

0 Externalities. 

A program of this kind will probably 
entail a range of negative externalities. 
For individual students, all three potential 
side-effects would be present. Certainly, 
total test-taking time would increase; 
evidence from New Jersey suggests, for 
example, that institutions now administer 
both the statewide Basic Skills test and 
their own local placement examinations. 

Moreover, many students will be denied 
their initial choice of classes or programs 
because of deficient performance; in 
New Jersey, more than a third of test 
takers are assessed as deficient in one 
or more basic skills each year. If 
deficiencies are detected, directed 
placement may significantly increase the 
amount of time it may take a student to 
obtain a degree. Major negative impacts, 
however, would likely be initially incurred 
in the form of faculty opposition and 
"teaching to the test." 

This overall pattern might be significantly 
changed if particular features of the 
proposed program were subject to 



modification. For example, changing the 
requirement that a single, statewide 
instrument be used would markedly 
reduce faculty opposition and teaching 
to the test. Negative impacts would 
remain in these areas, but would likely 
be not so strong. Allowing institutions to 
use their own instruments, moreover, 
would also reduce the burden of testing 
for individual students. It would not, 
however, provide a much payoff in 
accountability to the general public as 
would a program based upon common 
testing. 

Changing the requirement that students 
be tested on exiting from remediation 
would also shift the payoff pattern for the 
program as a whole. Because of a 
demonstrable association between 
directed placement and persistence, 
individual benefits would not be so 
strong as in the base program, and the 
benefits associated with assuring 
extemal constituents and the public that 
college students possess basic skills 
would evaporate. At the same time, such 
externalities such as teaching to the test 
and perceived violations of academic 
freedom would be all but eliminated. 

2. Requiring Institutions to 
Undertalce Local Assessment: 

As noted earlier, the center of gravity for 
most state-based assessment efforts has 
been to require institutions to design and 
implement their own local assessment 
programs. Current Board of Regents 
policy in Arizona is consistent with tliis 
thrust. While many variations in what is 
required are apparent across the eight 
states where such plans are in place, 
their general payoff pattern is presented 
in Figure 3. Once again, standard 
categories of benefit, cost, and 
externality are used. Notable features 
are as follows: 
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o Benefits. 

Most of the documentable benefits of 
this alternative are centered in the 
cuniculum-partlcularfy in improvements 
in cunicuiar structure and sequence. 
Benefits to individual students are 
certainly present, but documentable 
increases in l<nowledge and sl<ill will be 
difficult to daim. Some extemal benefits 
will also be present, but the primary 
focus on intemal evaluation and 
improvement will also imply that obtained 
infbmiation is less directly useful to 
extemal constituents. All these benefits, 
of course, will depend heavily upon what 
the institution in fact proposes to do. 
Absent information on graduate 
placement, for example, benefits 
assodated with consumer choice will 
largely disappear. Effective "packaging" 
of infomiation intended largely for 
intemal management, however, can also 
serve to demonstrate accountability to 
the public. 

o Costs. 

Overall costs for programs of this design 
are in the standard estimated range of 
$7-12/FTE student. Once again, 
however, the question of who should 
bear these sosts has a number of 
answers. Student fee alternatives, as 
practiced in South Dakota, seem 
comparatively inequitable because 
individual students do not generally 
receive proportionate direct benefit. 
Requiring institutions to fully absorb 
these costs as part of their existing 
budget, as practiced in Missouri, may 
require substantial reallocation and 
associated opportuni* costs. Where 
such programs are f y supported by 
additional dollars, as for example in 
Virginia and New Jersey, an additional 
burden is placed on the taxpayer, who 



may or may not receive commensurate 
payoff in the fbrm of accountability and 
improvements in general welfare. In 
practice, most states share such costs 
between institutions and taxpayers- 
-approprlating limited additional funds to 
support local assessment, or using 
existing categorical grant or non-base 
incentive funds to support a range of 
institutional efforts. 

o Externalities. 

The most important negative 
consequences associated with this 
alternative occur for faculty and for 
institutions. For faculty, a major 
consequences is a considerable 
investment of time in the design of 
assessment instruments and in scoring 
and interpreting the results of local 
examinations. Although each institution 
is free to choose its own approach, 
many faculty will also feel threatened in 
the initial stages of the program. Both 
these consequences will change 
depending upon the kinds of 
assessment instniments used: faculty- 
designed instruments will increase 
faculty burden and reduce opposition; 
commercial standardized tests (even if 
chosen by faculty themselves) will 
inaease the perceived threat but will 
substantially reduce direct faculty 
burden. For the institution, moreover, 
substantial costs may be incurred in 
establishing an additi( -"al administrative 
function. 

This pattern can also be substantially 
altered by changing one or more 
features of program design. For 
institutions, the payoff structure shifts 
markedly depending upon the 
percentage of direct costs that Is 
covered by additional resources. If the 
full cost of the pro jram must be 
absorbed, most institutions will see such 
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a program as a 'breakeven' proposition 
at best; most will therefore initially resist 
its adoption as state policy. For faculty, 
opposition and burden a'e directly 
afwwSd by the l<inds of approaches 
proposed-particularly the degree to 
which they rest upon standardized 
testing and the level of choice that 
faculty are ailowbd in choosin^ 
appropriate assessment instruments ^.^ 
techniques. Emerging experience seems 
dear on the point that scne degree of 
faculty opposition will be experienced no 
matter what kind of proQram is 
proposed. It is also clear, however, that 
the greater the degree of faculty 
Involvement, the more likely curricular 
benefits will be. 

For individual students, however, few 
features of the program's design will 
cause a shift in preference: most benefits 
are indirect, and because assessment 
results are not used to determine the 
fetes of individual stjdents, important 
extempllties are absent. The major 
exception here is cost, as given 
uncertain individual payoffs, it will be 
difficult to argue that individual students 
should bear a substaritial cost burden for 
this activity. 

These two cases can only briefly 
illustrate the manner in which the 
particular costs and benefits of any 
proposed asse&sment alternative for 
Arizona might be compared. Naturally, in 
the absence of a concrete policy 
proposal on assessment, they will be 
indicative only. At the general level, 
however, the framework provided can 
aid in both identifying the major 
categories of cost and benefit associated 
with any assessment approach and in 
roughly estimating the trade-offs among 
different posed alternatives. 
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Figure 1 

Benefits, Costs, and Externalities of Institutional Assessment 
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Figure 2 

its. Costs, and Externalities of Institutional As 
Case 1: Statewide Testing of Basic Skills 
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